-
Notifications
You must be signed in to change notification settings - Fork 133
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add a custom video processing cookbook for fitness training with Inference, Supervision, and GPT integration. #820
Conversation
Hi there, thanks for contribution, we do really appreciate that. Few comments and I believe we could move forward with this change:
Comments about the content of the notebook (may be treated as potential enhancements or just ignored):
|
Thank you very much for your feedback Paweł! Your comments and suggestions for potential enhancements are great. I will address all of them and make another PR shortly. |
is that you who is squating at the video? |
Yes, I took that video on purpose so that it wouldn't have any copyright issues. I added the --no-check-certificate since I have the video stored on google drive and in some cases the download without it didn't work. So if we could place the media on your hosting that would be a great solution for that! |
Hello Paweł, I just committed a new version with the changes you requested. Your suggestions for potential enhancements are fantastic! Unfortunately, I was only able to implement one of them—the butt wink detection and analysis. For this, I added left and right pelvic tilt angle detection and processing. The ideas for tracking the barbell's center of trajectory and incorporating different squat variations are excellent too. However, they would require quite a bit of additional code, which might push this notebook beyond the intended simplicity of a cookbook example. For now, I’ve decided to move forward with this version. That said, if any of those ideas are something you'd really like to see, please let me know, and I'll be happy to add them! Please let me know if you have any thoughts, questions, or concerns. Thank you so much! |
ok, that works - when I have some free time this week I will push the video on our hosting and then that would be simple wget to pull it- I will provide a link, then you can update a notebook and we are good to go |
Awesome, thank you so much Paweł! I'm ready to make the changes anytime. |
Ok, could you please just also - please attribute you as the author of the example in readme |
and could you also remove gif bin from the repository? I will squash merge such that there should not be a mark of the bins in git history |
ok, URL left to be changed and we are good to go |
Oh, I thought I changed a URL cell in my notebook, ie it is now I can see it in my most recent "Fixes" commit, is it not displaying for you there? I may be missing something here. |
a ok, did not get displayed in diff |
ok, we are good to go, I will merge and release tmr and will try to get a decent exposure. Thanks for contribution |
Awesome, thank you so much Paweł! I really appreciate all of your help and great advice! I'm glad I got to contribute! |
Description
Please include a summary of the change and which issue is fixed or implemented. Please also include relevant motivation and context (e.g. links, docs, tickets etc.).
This cookbook demonstrates a custom video processing use-case: creating a video-based fitness trainer. It uses Roboflow Inference to track body keypoints, calculate joint angles, and annotate them on video frames in real-time with Supervision. Additionally, it integrates GPT to provide fitness advice based on video frames and recorded data. The goal of this cookbook is to demonstrate how integrating Inference can enhance foundation models with vision capabilities like GPT 4o for more accurate, data-informed video processing.
List any dependencies that are required for this change.
openai, supervision: Installed automatically in the notebook.
Type of change
Please delete options that are not relevant.
How has this change been tested, please provide a testcase or example of how you tested the change?
This notebook was tested by running end-to-end on a sample fitness video. The outputs included annotated video frames with joint angles and GPT-generated fitness advice. Since the notebook runs independently, it does not affect other code or dependencies.
Any specific deployment considerations
For example, documentation changes, usability, usage/costs, secrets, etc.
Include a link to the cookbook in documentation for similar use cases (e.g., video processing, Yolo-Pose models, or GPT integrations).
Ensure users are aware of potential costs when using GPT and processing video data.
Docs